International Journal of Trend in Scientific Research and Development (IJTSRD) 

Volume 4 Issue 4, June 2020 Available Online: www.ijtsrd.com e-ISSN: 2456 - 6470 

> < 

Godeye: An Efficient System for Blinds 

Sabin Kbader^, Meerakrisbna M Ri, Resbma Roy^, Willson Joseph 

lUG Scholar, ^Assistant Professor, 

I'^Department of Computer Science, Sahrdaya College of Engineering and Technology, Thrissur, Kerala, India 


ABSTRACT 


The technology is growing vastly. Everyone in humanity has some limitations. 
One of those limitations is visual disability. So we are here with a system that 
helps the visually disabled people. The framework here contains object 
detection with voice assistance within an app and a hardware part attached to 
the blinds stick for distance calculation. The app is designed to support the 
blind person to explore freely anywhere he wants. The working of the 
framework begins by surveilling the situations around the user and 
distinguishing them utilizing a camera. The app will then detect the objects 
present in the input video frame by using the SSD algorithm comparing it with 
the trained model. The video captured is partitioned into grids to detect the 
object/obstacle. In this way, the subtleties of the object detected can be 
achieved and along with it distance measurement can also be calculated using 
specific algorithms. A Text to Speech [TTS] converter is utilized for changing 
over the data about the object detected into an audio speech format. The 
framework/application passes on the scene which the blind people is going in 
his/her territorial language with the snap of a catch. The technologies utilized 
here makes the framework execution effective. 

KEYWORDS: visual impairment, android phone, object/obstacle detection, text- 
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1. INTRODUCTION 

Real time object detection is vital for several real-world 
vision applications. Recent studies show that certain objects 
like a face can be detected reliably and fairly quickly. The 
speed of object detection isn't fast enough as needed by the 
applications. The goal of this paper is to supply a way which 
can help the visually impaired person to detect obstacles 
through object detection. One essential sense among human 
sense is vision and also vision plays an important role 
inhuman perception about the encompassing environment. 
Hence, thousands of papers are published on these subjects 
that propose a selection of computer vision products and 
services by developing new electronic aids for the blind. This 
paper aims to introduce a proposed system that restores a 
central function of the visual system which is that the 
identification of surrounding objects. Our contribution is to 
introduce the thought of a visible substitution system based 
on features extractions and matching to recognize/identify 
and locate objects in the video in real-time through an 
android app. 

Computer vision technologies, especially the deep 
convolutional neural network, have been rapidly developed 
in recent years. It is promising to use state-of-art, computer 
vision techniques to assist people with vision loss. The main 
aim is to find the possibility of using the sense hearing to 
know visual objects. It is not often realized by many people 
that we are capable of identifying the spatial location of a 
sound source just by hearing it with two ears. The work aims 
to guide blind people through the output of the system by 
voice to navigate them through an android app. The 
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methodology of this work includes Object Extraction, 
Feature Extraction, and Object Comparison. 

One of the issues faced by visually impaired individuals is to 
travel in a corridor. They cannot find either they need to turn 
left or turn right once they reach the end of the corridor and 
they get stuck in between the objects if it comes along their 
way. When the visually impaired individuals enter the 
corridor, they need to find the border of the sidewalk within 
the corridor. They use their walking stick to define their 
current location, the existing solutions somewhat helped 
visually impaired individuals. But all the solutions has its 
drawback. And there's no existing method that helps to solve 
the all basic needs of blind people. All the existing systems 
are designed just for a selected purpose. 

Although object tracking represents a fundamental problem 
in the computer vision community due to the wide range of 
related applications (e.g., automatic video surveillance, 
wearable assistive devices, robot navigation, structure from 
motion, or human-machine interaction], it is still an open 
issue of research. Despite significant progress achieved 
within the last decade, tremendous challenges still exist in 
designing a strong object tracker able to handle important 
changes in viewpoint, object motion, light variation, object 
occlusion, or background clutter. Through this paper, we 
propose a completely unique joint object 
detection/recognition frame-work, called GOD EYE, based on 
computer vision algorithms and offline trained deep 
convolutional neural networks. The proposed framework is 
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integrated within a device, dedicated to visually impaired 
[VI] users. Here we making an android app for the users. 

There are several object detection strategies/algorithms 
which will be used here like YOLO[2][21], SSD[11], 
RCNN[16], Fast R-CNN[9], Faster R-CNN[10], R-FCN[12]. 
These algorithms give different rates of performance. So, by 
considering the implementation simplicity and for improving 
performance comparing with other object detection 
algorithms SSD gives better performance and simple 
implementation. 

So, our contribution is an android app for visually impaired 
people using the better performing object detection 
algorithm with voice assistance and measuring the distance 
using an ultrasonic sensor which is helpful for the blinds in 
their day-to-day life like staying off from the obstacles and 
easily wandering in the new surroundings by the voice 
output and the alert signal. 

Here we are using the model SSD for object detection. Here 
we are employing a trained model SSD MobileNet VI coco 
model, so that we can implement the system godeye easily, 
the object detection algorithm used here gives a better 
performance than the other existing object detection 
algorithm as it follows the bounding box prediction method 
for detecting the objects. So we are using a better algorithm 
here for detecting the object efficiently and with great 
accuracy. 

The method that we propose here has the following 
specification: 

> Real time processing & output. 

> Better efficiency than other existing methods. 

> Easy to handle. 

> Alert signal after measuring distance. 

The object detection CNN i.e. SSD ensures real-time pro¬ 
cessing and this deep learning model method used here is 
much helpful in implementing the system easier than using 
other object detection algorithms and therefore the detected 
object will be given as voice feedback by using TTS[18]. The 
distance measurement and alerting also become easy by 
using the ultrasonic sensor. 

We summarize our contributions as follows: 

> We propose GODEYE, with an object detection app 
based on SSD, it achieves an accurate detection result. 

> We propose GODEYE, with a distance calculation and 
alerting system based on an ultrasonic sensor, it helps 
users by alerting from the moving obstacles. 

> The camera and ultrasonic sensor are attached to the 
blind stick. The camera will be connected to the app to 
detect the object and provides the voice assistance. 

II. PROBLEM DEFENITION 

Visually impaired people face many challenges and one 
among the main challenges faced by them is self-navigation. 
It's very difficult for them to travel through a corridor. This is 
one among the main challenge faced by them. They find it 
hard to locate the directions i.e. they can't find whether or 
not they need to either turn left or turn right once they reach 
the end of the corridor and that they get stuck in between 
the objects. While walking through the corridor they should 
find the border of the sidewalk. They use a walking stick to 


define the situation. Although the present arrangements 
helped them yet these arrangements have downsides. 
There's no existing solution that solves all the requirements 
for the assistance of a visually impaired person. The existing 
solutions designed for a selected purpose. Each solution 
solved only a selected problem. Those never solved the 
whole problems as an entire. 

III. RELATED WORK 

NETRA[3], the smart hand glove which acts as an electronic 
eye is among the related works. The glove detects the 
obstacles within the path of the visually impaired person and 
it warns them. It works as an artificial eye for the person. 
The glove extracts the text from any image that contains text 
and converts that into speech. Another feature of this glove 
is that it helps the people to spot the object around them. 
The glove sorts out all the issues faced by the people. All the 
above tasks are very essential for the safe navigation of 
blinds. 

NavBelt[4] is another related technology, this is often a 
guidance device that uses a mobile robot obstacle avoidance 
system. NavBelt is worn by the user like a belt and is 
equipped with an array of ultrasonic sensors. This will be 
utilized as both primary and secondary aid. This consists of a 
portable computer, ultrasonic sensors, and stereophonic 
imaging technique to process the signal from the ultrasonic 
sensors. Acoustic signals are provided via a group of stereo 
earphones which guides the user around obstacles or 
displays a virtual acoustic panoramic image of the traveler's 
surrounding. Acoustic signals are transmitted in the form of 
discrete beeps or continuous sounds. Experimental results 
with the NavBelt simulator and a portable prototype shows 
that users can travel safely in any unfamiliar and cluttered 
environment at speed of up to 0.8m/s. 

The walking cane or the guide cane is another related 
technology. It detects the obstacles on the way using IR 
sensors and produces a buzzing sound as a warning. Using 
walking cane the obstacles only up to the knee-level within a 
limited range of 2-3 feet can only be detected. The main 
drawback is that the buzzer's sound isn't audible within the 
traffic sounds on the roads[5]. 

The Voice-enabled smart walking stick is another related 
technology. Ultrasonic sensors. Pit sensing unit. Water 
sensing unit, GPS module, GSM module Voice synthesizer, 
relay, speaker, embedded system, and battery are used in the 
system. The ultrasonic sensors are used in different sensing 
units of the system. If the smart walking stick is made with at 
most accuracy, then it'll help the blind people to move from 
one place to a different without other's help. This could also 
be considered an unrefined way of giving the blind a sense of 
vision. The usage of the stick can reduce the visually 
impaired from relying on other family members, friends, and 
guide dogs while walking around. The proposed combination 
of various working units makes a real-time system. The 
position of the user monitored is the system and also 
provides dual feedback. This feedback makes navigation 
more safe and secure. Objects or obstacles ahead of the users 
are detected using the smart stick and warning voice 
messages are sent as feedback instead of vibrations. The 
automated room equipment switching incorporated within 
the stick are going to be useful while they're indoor. The 
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advantage of the system lies in the fact that it will prove to 
be a low-cost solution to many blind people worldwide[6]. 

Another related work is a smart navigational shoe. An 
electronic kit that is fixed in the shoe can be used by the 
blind or visually impaired person. The hardware kit consists 
of four vibrators, one Arduino, Bluetooth connection, and 
one battery. The obstacles along the path are sensed by 
sensors while moving along the path, if any obstacles 
encounter the path this will detected and informs the user. 
An android application that can be integrated with the smart 
navigational shoe is developed here. The starting point and 
the destination point are given as input by the user. The path 
for navigation is displayed using maps to users. The 
application which is integrated with shoes directs the 
person. The vibrators will vibrate along the path if an 
obstacle is detected. If it's a left turn that is to be taken 
according to the navigation, the vibrator assail the left side 
will vibrate. If it's a right turn that's to be taken according to 
the navigation, the vibrator set on the right side will 
vibrate [7]. 

A smart cap for the blind using the Raspberry Pi system is 
another related work that's easy to understand and 
maintain. This technique uses Raspberry pi which is a small 
processing device that works as a computer at a 
comparatively low cost. Visually impaired people find 
difficulties in detecting obstacles while walking in a street. 
By making use of Raspberry Pi, the system is intended to 
provide artificial vision and object detection, real-time 
assistance via GPS. The system consists of Ultrasonic 
sensors, GPS module, and therefore the feedback is received 
as audio. Voice output works through TTS (text to speech]. 
The proposed system detects an object around them and 
sends feedback in the form of speech that's warning 
messages via earphones. It can also provide navigation to a 
selected location through GPS. the general system aims to 
provide a low cost, efficient navigation, and obstacle 
detection aid for the visually impaired which provides a 
sense of artificial vision by providing them with information 
about the environmental scenario of the static and dynamic 
object around them so that they can walk independently[8]. 

IV. GODEYE 

The system is known as GODEYE. It includes an object 
detection app with voice assistance and Ultrasonic sensor for 
distance calculation and alerting. Normally, visually 
impaired people use a cane as a guide for them to protect 
themselves from obstacles. Most of the surrounding area are 
often covered with the assistance of the cane, especially the 
area nearer to their legs like stairs, etc. The system is 
designed to provide complete navigation for the user within 
the environment around them. It guides the user about 
obstacles, also it provides information about the appropriate 
obstacle-free path. This technique is an intelligent 
monitoring and alert system that involves the observation of 
real-time activities, with real-time constraints on video 
surveillance. The system consists of an android app 
connected to the camera attached to the blind's stick. This 
camera detects the obstacles/objects that come along the 
path of the visually impaired people or the users and an 
ultrasonic sensor attached to the blind stick measures the 
distance between the user and therefore the obstacle. The 
information is provided through voice output by the app and 
therefore the sensor gives an alert signal if the obstacle is 


extremely near to the user. Fig.l shows the diagram of the 
system. 



Fig.l. A block visualization of the system 


The main challenge is to get the response in minimum time 
and also that the efficiency and accuracy of detecting the 
object should be maximum and it should also guide the user 
correctly, to overcome these challenges, here we are using 
better technologies and algorithms. We are using the SSD 
algorithm for better object detection and accuracy and 
Ultrasonic sensor for measuring the distance. To alert the 
user we are using buzzer which produces a beep sound. On 
opening the app the video surveillance begins. The 
streaming video is provided as an input to the object 
detection algorithm. Using the object detection algorithm the 
obstacles/obj ects that come along the navigation path of the 
user are detected. The backend used for object detection is 
Tensor Flow, the object detection algorithm gives the output 
as a string and therefore the string is given as an input to the 
TTS[18]. The TTS provides a voice output to the user. The 
architecture diagram of the android app is shown in Fig.2. 

Simultaneously the distance calculation between the user 
and the obstacle is done using the ultrasonic sensor attached 
to the blind stick. For distance measuring the ultrasonic 
sensor is attached to Arduino UNO[19], and for the signal 
alert, a buzzer is attached to it. The space between user and 
object be calculated. A minimum distance is fixed, then the 
calculated distance and the minimum distance is compared if 
the distance between the user and the obstacle is less than 
the fixed minimum distance or equal to the fixed minimum 
distance the alert system gives a beep sound to the user 
which is inbuilt on the stick. The connections of distance 
calculation with a buzzer are shown in Fig.3. This how the 
GODEYE works. 

A. OBJECT DETECTION 

The detection of objects within the videos is done using 
feature extraction and detection. Feature matching using 
invariant features has obtained significant importance with 
its application in various recognition problems. These 
techniques have helped us to match images regardless of 
various geometric and photometric transformations 
between images. 

The visual substitution system here is based on the 
identification of objects around the visually challenged 
person using an android App connected to the camera 
attached to the blind's stick. This technique should find the 
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invariant characteristic of objects to viewpoint changes, 
provide the recognition, and reduce the complexity of 
detection. The method we've proposed here is based on the 
key-points extraction and matching in the video. A 
comparison is formed between the query frame and 
database objects to detect the objects in each frame. For 
every object detected a string file will be created and stored. 
Hence object detected string are going to be stored and given 
as input to TTS. 

The video will be loaded using the real-time camera for 
object detection; then feature extraction is done. SSD (Single 
Shot Detector] algorithm is employed here for object 
detection. When the app is opened the video surveillance 
begins. The streaming video is provided as an input to the 
object detection algorithm. Using the object detection 
algorithm the obstacles/objects that come along the 
navigation path of the user are detected. The object 
detection algorithm gives the output as a string and 
therefore the string will be stored. Then the object detected 
string is given TTS for audio/Voice output of the object 
detected. 

Here for object detection, we are using SSD300. By using 
SSD 3 00 [20], we only need to take one shot to detect multiple 
objects. It achieves a 74.3% mAP at 59 FPS. The network 
architecture of SSD300[24][25] is shown in Fig.4. to urge 
more accurate detection results, different layers of feature 
maps also are browsing the convolution for object detection. 
Each convolution will provides a specific number of 
bounding boxes for object detection. So, in total SSD300 
gives 8732 boxes in total which is more than that of 
YOLO[13][14][15]. Due to this much bounding box, object 
detection is accurate and efficient. The model used for object 
detection here is the ssd mobile net vl coco model. In SSD 
the image is divided into grids and each grid cell is 
responsible for object detection in that region. By dividing 
into grid cells there maybe loss occurs while detection, based 
on this the loss function of SSD is defined as: 

L (x; c; 1; g] =i(Lconf (x; c] + Lioc(x; 1; g]](l] 

N 

Where L is loss function [20], Lconf is the confidence loss, Lioc 
is localization loss, is the weight for the localization loss, 1 is 


predicted box, g is ground truth box, N is the matched default 
boxes, c is the multiple class confidences and x is the 
indicator for matching default box with the ground truth box. 

Based on the above object detection algorithm technique the 
object detection is implemented in-app. In this, the input 
frame must be 300x300. The SSD300 gives a faster and 
accurate results of object detection. The object detection is 
the major a part of the GODEYE system for visually impaired 
people. 

B. TEXT-TO-SPEECH 

The Google Text-to-Speech API is employed for getting the 
voice output of the detected object. The objects detected by 
class predictions in every frame will be a string e.g. "cat" and 
it'll be stored. This string will be given as input to the TTS 
converter. The TTS converter will convert the string that got 
into the user's natural language using the gTTs package used 
in the system. The TTS converter will do text analysis to the 
string given as input for converting it to the user's natural 
language. After the text will be searched in the speech 
database and choose the units compared to that. The speech 
generation module that presents within the TTS converter 
will make the text/string into speech and given as speech. 
This TTS process is completed by a text-to-speech 
synthesizer. The general process steps are shown in Fig.5. 
The speech made up of the input string will be given as the 
output from the android app. this is the main part of the 
implemented system. If multiple objects detected from one 
frame then based on the accuracy of the object the speech 
out will be got on the user's natural language. The speech 
from the TTS[22] is the major output from the app for the 
visually impaired person and it's heard by the user using 
earphones or by the phone speaker[18]. 

C. DISTANCE MEASUREMENT 

The ultrasonic sensor is employed here for distance 
measurement between the user and the object/obstacle[23]. 
The ultrasonic sensor[19] attached to Arduino UNO are 
going to be attached to the blind's stick. The ultrasonic 
sensor one eye produces waves to detect the object, when 
the waves hit the object the waves will reflect the opposite 
eye of the ultrasonic sensor shown in Fig.6 by that the space 
is measured 
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Fig.2. The system design of Android App. 
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Distance Measurement 

Fig.3. The connections of distance calculation and alert. 

between the thing and therefore the user. The Arduino IDE is employed here for coding the ultrasonic sensor. It uses embedded 
C for coding the sensor, within the sensor, we set a threshold distance so that when the obstacle/object is equal or less than the 
threshold distance then an alert signal will be given to the user. The ultrasonic sensor can detect only the distance from 2cm to 
400cm of non-contact measurement functionality with a ranging accuracy which will reach up to 3mm. But it's a sufficient 
distance for the user to get alerted. The ultrasonic sensor will update the distance between the user and the obstacle during a 
specific interval of time. So that the alert system can alert the user/visually challenged person on reaching the threshold 
distance. 

The ultrasonic sensor uses sonar waves for echolocation, which be able to measure the distances. The ultrasonic sensor module 
has 4 pins- Ground, VCC, Trig, and Echo. The VCC and Ground pins of the ultrasonic sensor need to be connected to the 5 volt 
and ground pins on the Arduino Board and the echo and trig pins to any Digital I/O pin on the Arduino Board. The distance 
between the sensor and the object is defined as: 

distance = speed x time 
s = vt [2] 

The speed of the sound above water level is 340 m/s or 0.034 cm/ps. From the Echo pin, we'll get the time duration which will 
be double because the wave must travel forward and return backward, to get the distance in cm, we need to multiply the 
received duration time from the echo pin by 0.034 and divide it by 2. so that the correct distance between the sensor and object 
is defined as: here V = 340 m/s or v = 0.034 cm/ps, so, 
s = 0:034 t=2 (3] 

This is how distance measurement is done by the ultrasonic sensor present within the system. Based on the distance measured 
the threshold distance will be set and on that basis, the alert signal is given to the user by using buzzer with a beep sound. The 
alert will continue if the user is moving closer to the object, the major contribution from the distance measurement system is: 

> Measure distance between user and objects. 

> Give alert signal to user based on the distance measured. 

V. SPECIFICATIONS 

Android Mobile phone, Arduino UNO, Ultrasonic sensor. Buzzer, Earphone, and White cane. 


Extra Feature Layers 



Fig.4. The system design of SSD.[20] 
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Fig.5. The system architecture of Text-To-Speech.[22] 


Distance Measured 



Fig.6. Distance measurement using ultrasonic sensor. 


VI. EXPERIMENTAL EVALUATION 

By evaluating the experiment here the prediction time taken 
by the object recognition android app is 70ms to 500ms. The 
distance calculation and cautioning was given elite with a 
deferral of 10ms. The threads given in the app makes the 
class expectation simpler. 

Table 1 shows the comparison of various systems that is 
related works and the implemented work. From the 
comparison table we can see that the system we proposed 
here is effective and performs superior to other/existing 
systems. The above comparison table shows that GODEYE is 
a very efficient system for blinds. 


TABLE I COMPARISON OF VARIOUS SYSTEMS 


Systems 

Power 

Consumption 

Range 

Response 

time 

NETRA 

Low 

Medium 

Slow 

Nev-Belt 

Medium 

Low 

Slow 

Smart Cap 

Medium 

Low 

Fast 

Smart Shoe 

Medium 

Low 

Slow 

GODEYE 

Low 

High 

Fast 


While calculating distance for this experiment the system is 
associated with the serial monitor of arduino IDE to check 
the outcome exactness and the alarming sign accuracy. At 
the point when the distance calculated arrived at the 
threshold distance then the alarm sign will begin at that 
point onwards with a particular time stretch. It proceeded 
until the user or object moves farther. The distance 
calculation yield to the serial monitor of arduino IDE is 
appeared in Fig.7. 
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Fig.7. Measured distance from user to object using 
ultrasonic sensor 

The output of the system proposed here is an audio output of 
the object detected and the alert signal that is beep sound 
that occurs if the distance calculated not exactly or 
equivalent to the threshold distance. The experimental 
evaluation shows that the framework executed is productive 
and superior to the existing system for visually disabled 
people. 

Vll. CONCLUSION AND FUTURE WORK 

Here, we implementing a system GODEYE. A system for 
visually disabled people. This system is introduced to 
overcome the difficulties faced by the visually disabled. Our 
system consist of two parts: Object detection with voice 
assistance and Distance measurement. SSD300 is used for 
objection detection. This strategy is favored due to its 
productivity, exactness, and accuracy. This makes the object 
recognition more faster, so the obstacles can be easily 
located by the user. Voice feedback is provided after object 
detection. This is done by the text-to-speech conversion 
using a text-to-speech converter, i.e. text-to-speech API. The 
distance between the obstacles and user is calculated using 
an ultrasonic sensor joined to the stick. The ultrasonic 
sensor is used to calculate the distance. These features are 
clubbed in our system. This makes our system efficient and 
enables self-navigation for the visually impaired. We feature 
the presentation of our framework as far as the speed of 
recognition with exceptional proficiency in distance 
calculation and high accuracy of detection. 


In the near future, the framework can be modified with SSD- 
MSN technology. Currently, SSD300 is used. With the usage 
of SSD- MSN several images frames can be detected. Nowthe 
distance calculation is done using the ultrasonic sensor. In 
the future, the distance measurement can be advanced by 
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measuring the distance using the camera. By including these 

progressions to our framework the framework transforms 

into a high performance and efficient app. 
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